Communication Bottlenecks in Scale-Free Networks 
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We consider the effects of network topology on tfie optimality of packet routing quantified by 7c, 
the rate of packet insertion beyond which congestion and queue growth occurs. The key result of 
this paper is to show that for any network, there exists an absolute upper bound, expressed in terms 
of vertex separators, for the scaling of jc with network size A^, irrespective of the routing algorithm 
used. We then derive an estimate to this upper bound for scale-free networks, and introduce a novel 
static routing protocol which is superior to shortest path routing under intense packet insertion 
rates. 
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Communication has stepped into a new era with the 
advent of the Internet, making possible information ex- 
change/transport across the globe virtually in an instan- 
taneous fashion between any two people who have ac- 
cess to it. Broadcasting and advertising messages, home- 
pages, blogs and practically any information posted on 
the WWW is within the reach of anyone accessing those 
pages and thus, downloading that information. This ac- 
tivity, exponentially increasing over the past years in- 
volves an incredible amount of information stored and 
transmitted through the physical infrastructure of the In- 
ternet, every second of the day. As the number of com- 
puters and users surpasses into the billions, one might 
naturally ask about the ultimate limits to using the In- 
ternet. In terms of transmission latency the Internet is 
pretty good already. As an illustration, consider the dis- 
tance between Los Alamos and Boston (as the crow flies) , 
which is about 3109 km. The speed of light in fiber 
is about 2/3 of that in vacuum, about 2 x 10^ km/ s. 
Thus the round-trip time for information between Los 
Alamos and Boston is about 31 ms. Performing a ping 
on a Los Alamos computer to a computer at Boston 
University gives for the round-trip time about 64 ms 
which is within a factor of two of the absolute physical 
bound. Therefore, no order of magnitude improvements 
can be expected in transmission latency for the Inter- 
net. The current paradigm in communication on net- 
works is packet switching where the message is divided 
into packets which are then routed between nodes over 
data links, independently from each other, and reassem- 
bled at the destination into the original message. This 
decentralized methodology makes information transmis- 
sion efficient by providing better utilization of the avail- 
able bandwidth (a single link can be used to transmit any 
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packet). However, due to the increasing demand of infor- 
mation carried through the Internet, delays can occur in 
packet delivery, mainly caused by device (end-user and 
router) latency. Device latency is the amount of time r 
that a device needs to process a single packet. Although 
the devices are getting better in their latency, this is a 
physical constraint and can never be completely elimi- 
nated. Since more packets may arrive at a node than 
it is able to process per unit time, queues can accumu- 
late and thus routers must have a storing capacity as 
well. These queues will naturally slow down information 
transport over the network. As an interesting observa- 
tion, the US Postal Service is capable of achieving higher 
information transmission rates than the current Internet. 
For example, for a Tl line which transmits at about 1.544 
Mbit/s, downloading a 4.7 GB DVD takes about 6.76hrs. 
If one ships 1000 DVD-s from coast to coast in the US, it 
will take about 3 days, but the transmitted information 
would have a bandwidth matching that of 94 Tl lines. 
Precisely this fact is exploited by DVD rental delivery 
companies like Netflix which distributes about 1.5 Ter- 
abytes of data per day, the same order of magnitude as 
the Internet Q. 

In spite of technological advances, the Internet is be- 
ing driven closer to its capacity. These facts lead us to 
two important questions: (1) How can one characterize a 
packet switched communication network's ultimate car- 
rying capacity? and (2) What routing algorithms will 
achieve this ultimate capacity? 

In this Letter we present a proof-of-principle study to 
show that the ultimate carrying capacity is strongly influ- 
enced by the network's structure. We demonstrate the 
existence of a solely topology determined upper bound 
7t for the congestion threshold 7c |2| which is the packet 
insertion rate at which queuing and congestion in the 
network appears. 

It has been conjectured that the degree distribution of 
the Internet follows a power law on several levels |3>0.I3- 



Recent experimental studies have strengthened the vahd- 
ity of this conjecture |y,l3l- ^'^^ o^^" study, we will confine 
ourselves to the the configuration model (CM) |1] which 
is one of the simplest models to generate a random graph 
with a power law degree distribution. The approach pre- 
sented here is, however, applicable to arbitrary graph 
structures. 

We consider all time scales measured in units of router 
latency r which for simplicity, we take to be unity. We 
will also assume that routers have infinite storage capac- 
ity. 

The Static Routing Problem. Denote by G{V, E) the 
physical substrate graph (network) for communication 
which we assume to be singly connected. Once a packet 
entering node s reaches its destination node d, it disap- 
pears from the system. The sequence of nodes and edges 
the packet visits constitutes the route for that source- 
destination pair. For a network of size iV, the routing 
problem consists of finding an assignment of routes for 
all N{N — l)/2 pairs of nodes. We shall call such an 
assignment set a Static Routing Protocol (SRP). 

We consider a previously studied 0, la 11^; 113 model 
of communication, which was motivated by the need to 
study the problem of congestion on the router-level In- 
ternet. Here, the packet transmission is modeled by a 
discrete time parallel update algorithm. At time t and 
at every node, a packet enters with probability < 7 < 1. 
The packet has a destination node, chosen uniformly at 
random from the remaining A'^ — 1 nodes. Every node 
i maintains a set of all packets that were sent to it by 
its neighbors in the previous step, eliminates from this 
set newly arrived packets whose destination was i, adds 
to this set the freshly injected packet (if there is one) 
and finally places elements of this set in a sub-queue in 
a random order. This randomization is needed because 
times are not resolved below the single-packet process- 
ing timescale, r. The sub-queue is then appended to the 
existing queue, if there is one, from before the i-th step. 
The top packet in the queue is then sent to a neighbor 
on G following the SRP. 

There is a critical rate 7c of packet creation at which 
there is an onset of congestion, i.e., abov e 7r ., pack- 
ets start accumulating on the network [3, [ij. This 
is commonly designated as the "congestion thresh- 
old". In Fig. O we show a rescaled version Q 
of the rate of steady-state packet growth ^(7) = 
limt^oo [n(i -I- At) — n{t)] / {NjAt) as function of 7 for 
both the shortest path (SP) protocol and the novel one 
proposed in this paper. Here n{t) is the number of pack- 
ets on the network at time t. This threshold can be ex- 
pressed in terms of the maximal node betweenness B for 
a given SRP. The betweenness b for a node is the number 
of SRP routes passing through that node. The highest 
among the N betweenness values (one for each node) 
resulting from the SRP is the maximal node between- 
ness B. For a given SRP route between a source s and 
destination d the average packet current incurred from 
the source at s is 'y/{N — 1). For a node with between- 
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FIG. 1: The relative sizes of the betweenness values intro- 
duced in the text. 



ness b the average packet inflow current will be given by 
bj/{N — 1). Since the outflow of packets occurs at unit 
latency, we will have queueing and congestion at the node 
for which this quantity reaches unity for the flrst time, 
namely at the node with b = B. Thus 
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For SP routing J3, M, U^ IHI j the node betweenness be- 
comes identical to the familiar, shortest path between- 
ness, B^^ [l^l- From Eq. ^ follows that for a given rout- 
ing protocol, the dependence of the congestion threshold 
7c on TV, is determined by the scaling with N of the 
maximal node betweenness B. Therefore, the best rout- 
ing protocol from the point of view of router congestion 
avoidance, should be the one for which B exhibits the 
slowest growth with N. Although there have been pre- 
scribed ad hoc adaptive protocols [Ij, llJ, uM that in- 
crease 7c, the above issue has not been systematically 
addressed. 

Next, we show that there is a lower bound Bt < B 
(and thus 7 < 7t) induced only by the topology of the 
network G, and it is independent of the routing protocol 
used. In other words, no SRP can do better than 77. 
This ultimate threshold Bt is essentially a communica- 
tion bottleneck quantifier for a given graph G. Among all 
possible SRPs (whose set is denoted as V), let us write 
Bopt for the smallest maximal betweenness value, namely 
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Bopt = mmsRPevB^"^, so Bt < Bopt (Fig. HJ. It is 
an open question whether the topological bound can be 
achieved by a routing protocol. Similar considerations 
have been made in the context of edge betweenness in 
Refs . [10.1171 . Here we focus on scaling of the bound Bt 
as function of A^. 

We introduce Bt using graph partitioning arguments. 
Given an arbitrary network G, partition the set of all 
nodes V into three non-empty sets denoted. A, X and 
B. Since G is singly connected, there will be edges run- 
ning between at least two pairs of the three possible pairs. 
Choose set X such that there are no edges running di- 
rectly between A and B in which case X is called a vertex 
separator. For any SRP we must designate a route for 
all pairs of nodes, therefore also for those pairs for which 
one node is in A and the other in B. Since X is a sep- 
arator set, all routes from A to B, must go through the 
nodes in X. Therefore, there are at least |^||i3| routes 
passing through X for any SRP. Since the maximum is 
always larger or equal than the average, the maximum 
betweenness incurred on the nodes in X can be no less 
than |][| ' . We define the sparsity [13] of the separator 




sparsity Qx* is determined by 



FIG. 2: Bipartitioning the graph into two vertex subsets A 
and B such as to obtain two vertex separators, c{A) and c{B), 
see text. 
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X the quantity Qx = \a\\b\ • Thus, associated with ev- 
ery vertex separator X there is a quantity Bx — 1/Qx 
providing a lower bound to the maximal betweenness on 
nodes in X. Let us denote by M the set of all possi- 
ble vertex separators in G. If we systematically consider 
all possible choices of vertex separators X 6 Al, we can 
find (at least) one separator X* for which Bx = l/Qx 
achieves its maximal value defined as Bt- Thus, the 
topology of the graph constrains the maximal between- 
ness to be no less than Bt, and for arbitrary routing, 
B > Bt — l/Qx* — l/minxGA^ Qx- Finding minimal 
sparsity vertex separators is an NP-hard problem [19j, 
and we shall not deal with it here. 

Due to the analytical and the computational difficulty 
in determining Bt, we focus on obtaining an analyti- 
cal estimate Be to Bt, and derive its scaling with N 
for random, uncorrelated, scale-free networks. This es- 
timate, while possibly being greater than the true topo- 
logical bound Bt, nevertheless provides a comparative 
value dependent only on the network topology. This es- 
timate, Be, allows us to quantify the performance of the 
SP protocol. 

We start by systematically considering every possible 
vertex separator in the graph as follows. First, bipar- 
tition the graph as shown in Fig. [21 into sets A and B 
with \A\ < \B\. Let c{A) be the subset of nodes in A 
which are adjacent to at least one node in B and let 
c{B) be the subset of nodes in B which are adjacent to 
at least one node in A. We can now obtain a vertex 
separator c{A) which separates sets A \ c{A) and B, or 
similarly, a vertex separator c{B) which separates sets 
B \ c{B) and A . Thus, going through all possible bipar- 
titions of the graph with | A| < N/2 ensures that we have 
considered all possible vertex separators of the graph. 
If c{A) is chosen as the separator then the sparsity is 
QeiA) = \c{A)\/i\A-c{A)\\B\) > \c{A)\/i\A\\B\). We 
obtain a similar expression for Qc(b) if c{B) is chosen as 
the vertex separator. Therefore 
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Next we use the notion of edge expansion Xe defined be- 
low. For a bipartition of the graph G into sets A and B, 
denote the number of edges simultaneously adjacent to a 
node in A and B as Ce{A, B). Then 
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and an edge expander graph has Xe ^ 0(1). Next con- 
sider a bipartition of the graph into A and B, and let 
1^1 = ciV" where c is a constant and < a < 1. From 
the edge expansion property of scale-free graphs with 
k-min > 3 US J the number of cut edges between A and B 
is at least XecN" — 0(iV"). We can bound from below 
both \c{A)\ and \c{B)\ (as needed by ©) by the minimal 
size m of the set of nodes that can contribute XecN°' cut 
edges. The size m is obtained by taking all nodes with de- 
gree higher than k, such that X Jj^ kP{k)dk = XecN"^, 
where P{k) = Ak~^ is the degree distribution of the 

l-Q 

graph. This yields k ^ N ^-^ . Therefore the minimal 
size of the set of nodes that can contribute XecN" edges 
is: 771 = A^ f^ P{k)dk ~ N ■ Ar(i-^)3^ and therefore, 
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The quantity m is bounded below by 0(1). For a given 
A we see that when a = 1 or in other words sets A and B 
in the bipartition are both 0{N), we get m = 0{N). For 
all other values of a, we get m < 0{N°'). As a decreases 
from 1, m also decreases until it becomes 0(1) and this 
occurs for the first time when a — 1/(A — 1). Thus, from 

(O and Q we get Qx' > O (n^^j and so 



Bt < Be = O N 



Since |^| < N/2, \B\ = 0{N), and a lower bound for the 
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From © we see that when A — > 2, we get the worst 
possible scaling of Be = 0{N'^), which can be under- 
stood from the fact that the graph becomes increasingly 
star-like, and for such a graph the central node trivially 
has B = 0{N'^). On the other hand, when A — > oo. 
Be -^ 0{N). In this case the graph approaches a ran- 
dom regular graph and random regular graphs are good 
vertex expanders |2ll |. This implies that for any biparti- 
tion into A and B, there exists a constant /x such that 
\c{B)\ > n\A\. Thus \c{A)\ > j^\A\, so \c{A)\ and 
\c{B)\ are linear in \A\ and hence Be = 0{N). 

When 2 < A < 3, for the networks generated by the 
configuration model to be uncorrelated requires that the 
maximum degree in the network Kmax ~ A^^^^ [22 |. 
Incorporating this upper cutoff in the arguments made 
above, we obtain Qx* > 0{N^) and hence Bt < Be = 
0{N2) (same as for A = 3 in ||HJ) ). From the inset in 
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FIG. 3: Numerical comparison for the performance of SP 
and hub avoidance HA protocols on a scale-free graph of size 
A'^ = 10^ and A — 2.5. The black circles correspond to the SP 
protocol and the red squares correspond to the HA protocol. 
The congestion threshold 7c beyond which packet growth oc- 
curs (^(7) > 0), is higher for the HA protocol as compared to 
the SP protocol. The inset shows that maximum betweenness 
for SP and HA protocols on scale-free graphs has power-law 
scaling with system size. The maximal betweenness B^ 
resulting from the HA protocol has scaling exponent 1.48, 

close to our estimate for the topological bound on the maxi- 

3 
mal betweenness Be ~ A2 . However, the maximal between- 
ness B resulting from the SP protocol grows much faster, 



Fig 3, we see that the scaling the maximal betweenness 
incurred by the SP protocol, B^^ ~ N^-^'^. This is much 
worse than the scaling of Be, and therefore suggests that 
an SRP for which the maximal betwenness scales like B,, 
would have a better performance than the SP protocol 
from the point of view of congestion. The question arises 
whether B^ can be achieved by any static routing proto- 
col. We answer this question affirmatively by presenting 
next an SRP for which the scaling of the maximal betwen- 
ness is superior even to the scaling of B^ and therefore 
significantly better than the scaling of B^^ . 

Our derivation of B^ suggests that the sparsity is small- 
est when obtained from a bipartition where the smaller 
set is of size of the order of the maximal degree. This 



suggests that, topologically, the betweenness for hubs is 
high, and using the SP protocol increases this between- 
ness since shorter paths largely tend to use hubs. More- 
over, using the SP protocol leaves a large number of al- 
ternate paths unused for routing. Exploiting these ob- 
servations, we obtain a novel SRP, which we call the hub 
avoidance (HA) protocol, as follows: (1) Remove x of the 
highest degree nodes. The network could now consist of 
several disconnected clusters. In every such cluster, as- 
sign a routing path for every pair of nodes using SP. (2) 
Place back the removed nodes with their edges. For ev- 
ery pair of nodes which have not been assigned a routing 
path in Step 1), assign one using the SP protocol. For our 
simulations we have chosen x — O.OIA^, but for optimal 
performance the functional dependence of x on N may 
be different. A detailed theory for this protocol with 
these considerations will be presented elsewhere. Here 
our primary purpose of presenting the HA protocol is 
to indicate that there exists an SRP for which the scal- 
ing of the maximal betweenness not only achieves, but 
surpasses the scaling of the topological estimate B^., and 
therefore is a significant improvement over the SP pro- 
tocol. This improvement comes from utilizing available 
alternate paths which, while not significantly longer than 
the shortest path, also considerably alleviate the load on 
the hubs. The plot in Fig. 3 shows the improvement in 
performance achieved by our protocol as reflected by the 
increase in the position of the congestion threshold and 
the lowering in the number of accumulating packets at a 
given packet creation rate 7 as compared to the shortest 
path protocol. 

Thus, in summary, we identify a bound to communi- 
cation arising purely due to the network topology and 
utilize this to show that there exist better SRPs than the 
SP protocol for routing on scale free networks. 
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